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DETAILED ACTION 
Response to Amendment 

Applicant's submission filed on 9/19/2005 has been entered. Claims 1-6 are pending in 
the application. 

Response to Arguments 
Applicant's arguments filed September 19, 2005 have been folly considered but are moot 
in view of the new ground(s) of rejection of the claims 1-6 based on Loui et al. U.S. Patent No. 
6,636,648 (hereinafter Loui), in view of Narayanaswami et al. U.S. Patent No. 6,504,571 
(hereinafter Narayanaswami) in view of Wilcock et al. U.S. Patent No. 6,741,864 (hereinafter 
Wilcock). 

As set forth in the present Office Action, the cited references teach the claim limitations 
set forth in the base claims. 

Loui's query using the query metric meets the claim limitation of calculating a distance 
between the photography location of the image data set and a predetermined reference position, 
based on the location data. It can be seen from Loui that the query for the images to be shown in 
a photo album requires a predetermined event such as the predetermined photography location 
for the query and the images found after query meets the criteria set forth in the query metric in 
which the distance between the queried image and the predetermined photography location is 
within the event boundary. 

This is manifested as follows. Loui teaches in column 9-1 1 adding annotations for 
pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
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arranges the picture according to an event or a sub-event wherein the event or sub-event are 
defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some threshold and the event segmentation stage 
determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value forjudging the distance 
between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries. 

Moreover, Narayanaswami discloses the claim limitation of calculating a distance 
between the photography location of the image data set and a predetermined reference position, 
based on the location data. Narayanaswami discloses a query based on the geographic boundary 
data such as the latitude, longitude and altitude data for a region of interest , or query for the 
pictures taken along the route as specified in the query language wherein the predetermined 
photography reference location is determined in the query language as within the geographic 
boundary or along the route as specified and the images being queried meets the criteria set forth 
in the query language. Moreover, the threshold is related to the ranges set forth in the query 
language for specifying the boundary for the region of interest, the altitude ranges such that 
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pictures taken within certain altitude ranges are still fall into the category of the region of interest 
(See Narayanaswami column 1 1, lines 21-67 and column 13, lines 1 1-63). 

Therefore, taking the combined teaching of Loui and Narayanaswami, it would have been 
obvious to have incorporated Narayanaswami 's teaching into Loui to calculating a distance 
between the location of a query image and the location for the query as a predetermined 
reference position based on the GPS location data embedded in the picture images as both 
references discloses location based query for images/pictures to be arranged on a page according 
to the event such as the region of interest or the route of interest. One of the ordinary skill in the 
art would have been motivated to do this to classify the images in a album according to the 
location of interest ( Narayanaswami column 1 1, lines 21-67 and column 13, lines 1 1-63 V 

It remains to show whether that Loui and Narayanaswami implicitly teaches judging 
whether the distance is over a predetermined threshold value. Loui that the query for the images 
to be shown in a photo album requires a predetermined event such as the predetermined 
photography location for the query and the images found after query meets the criteria set forth 
in the query metric in which the distance between the queried image and the predetermined 
photography location is within the event boundary. The event boundary for an event occurs 
within a region of interest, for example, a park or a city requires the calculation of the distance 
between the locations for the images being queried and the query location of interest and the 
distance should be less than a threshold such as the zero threshold for exact query or within a 
predetermined threshold for approximate query in a query language. 

Nevertheless, Wilcock explicitly discloses using threshold for classifying the pictures 
within a location of interest and therefore Wilcok discloses the claim limitation of judging 
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whether the distance is over a predetermined threshold value ( See Wilcok column 7, lines 28- 
51). 

Therefore, taking the combined teachings of Loui, Narayanaswami and Wilcock, one of 
the ordinary skill in the art would have incorporated Wilcock' s thresholding into Loui and 
Narayanaswami' s method because these references deal with the image queries based on the 
location of interest using a criteria set forth in the query wherein Wilcock specifically uses a 
threshold number to classify the groups of pictures. One of the ordinary skill in the art would 
have been motivated to use the thresholding to specifically classify the pictures when they fall 
into a number of categories of events ( See Wilcok column 7, lines 28-51 ). 

Re claims 3-5, 8, 10, 13, 15, 18 and 20, Loui teaches in column 8-1 1 adding annotations 
for pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
arranges the picture according to an event or a sub-event wherein the event or sub-event are 
defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some threshold and the event segmentation stage 
determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value forjudging the distance 
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between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries, e.g., the distance is either zero or 
larger than a threshold value so that the photo is judged to be either within or outside the event 
and sub-event boundaries. Loui further teaches in column 7-1 1 the page layout module presents 
to the user an automatically laid out album organized by event with the algorithm choosing the 
albuming parameters by default or specified by the user. Therefore, Loui further discloses the 
claim limitation of "obtaining related data, which is related to the photography location of the at 
least one image data set for which the distance is over the predetermined threshold value, based 
on the location data attached thereto, from a related data storage means which stores a plurality 
of related data sets" wherein the related data set is the metadata including the location of 
photographing stored in the database. Loui further discloses the claim limitation of "generating 
album data, which includes the related data" because Loui discloses displaying album organized 
by event wherein the event are associated with the pictures in the album. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1,3-5, 6, 8, 10, 11, 13, 15, 16, 18, 20 and 21-24 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Loui et al. U.S. Patent No. 6,636,648 (hereinafter Loui), in 
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view of Narayanaswami et al. U.S. Patent No. 6,504,571 (hereinafter Narayanaswami) in view of 
Wilcock et al. U.S. Patent No. 6,741,864 (hereinafter Wilcock). 

Re claims 1,6, 11, and 16, Loui discloses a method for generating an album based on 
album data including at least one image data set, which has been photographed during a trip and 
which has location data representing a photography location attached thereto, comprising the 
steps of: 

Calculating a distance between the photography location of the image data set and a 
predetermined reference position, based on the location data (e.g., Loui teaches in column 9-11 
adding annotations for pictures and associating metadata to the pictures. Loui teaches an 
albuming algorithm that arranges the picture according to an event or a sub-event wherein the 
event or sub-event are defined by the metadata including metadata tags and GPS information to 
determine location and subject depth information. The information defines the context 
surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event. Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
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reference position associated with an event or sub-event such as the event and sub-event 
boundaries)', 

Judging whether the distance is over a predetermined threshold value (e.g., Loui teaches 
in column 8-11 adding annotations for pictures and associating metadata to the pictures, Loui 
teaches an albuming algorithm that arranges the picture according to an event or a sub-event 
wherein the event or sub-event are defined by the metadata including metadata tags and GPS 
information to determine location and subject depth information. The information defines the 
context surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event. Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries)', 

Classifying the image data set according to the result of judgment {e.g., Loui teaches in 
column 8-11 adding annotations for pictures and associating metadata to the pictures. Loui 
teaches an albuming algorithm that arranges the picture according to an event or a sub-event 
wherein the event or sub-event are defined by the metadata including metadata tags and GPS 
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information to determine location and subject depth information. The information defines the 
context surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event. Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries, e.g., the distance is either zero or larger than a threshold value so that the photo is 
judged to be either within or outside the event and sub-event boundaries)', 

Generating the album data according to the result of classification {e.g., Loui teaches in 
column 7-11 the page layout module presents to the user an automatically laid out album 
organized by event with the algorithm choosing the albuming parameters by default or specified 
by the user). 

However, it remains to be determined whether Loui's query using the query metric meets 
the claim limitation of calculating a distance between the photography location of the image data 
set and a predetermined reference position, based on the location data. It can be seen from Loui 
that the query for the images to be shown in a photo album requires a predetermined event such 
as the predetermined photography location for the query and the images found after query meets 



Application/Control Number: 10/669,593 Page 10 

Art Unit: 2672 

the criteria set forth in the query metric in which the distance between the queried image and the 
predetermined photography location is within the event boundary. 

This is manifested as follows. Loui teaches in column 9-1 1 adding annotations for 
pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
arranges the picture according to an event or a sub-event wherein the event or sub-event are 
defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some threshold and the event segmentation stage 
determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value forjudging the distance 
between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries. 

Moreover, Narayanaswami discloses the claim limitation of calculating a distance 
between the photography location of the image data set and a predetermined reference position, 
based on the location data. Narayanaswami discloses a query based on the geographic boundary 
data such as the latitude, longitude and altitude data for a region of interest , or query for the 
pictures taken along the route as specified in the query language wherein the predetermined 
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photography reference location is determined in the query language as within the geographic 
boundary or along the route as specified and the images being queried meets the criteria set forth 
in the query language. Moreover, the threshold is related to the ranges set forth in the query 
language for specifying the boundary for the region of interest, the altitude ranges such that 
pictures taken within certain altitude ranges are still fall into the category of the region of interest 
(See Narayanaswami column 11, lines 21-67 and column 13, lines 1 1-63). 

Therefore, taking the combined teaching of Loui and Narayanaswami, it would have been 
obvious to have incorporated Narayanaswami' s teaching into Loui to calculating a distance 
between the location of a query image and the location for the query as a predetermined 
reference position based on the GPS location data embedded in the picture images as both 
references discloses location based query for images/pictures to be arranged on a page according 
to the event such as the region of interest or the route of interest. One of the ordinary skill in the 
art would have been motivated to do this to classify the images in a album according to the 
location of interest ( Narayanaswami column 11, lines 21-67 and column 13, lines 1 1-63 ). 

It remains to show wheteher that Loui and Narayanaswami implicitly teaches judging 
whether the distance is over a predetermined threshold value. Loui that the query for the images 
to be shown in a photo album requires a predetermined event such as the predetermined 
photography location for the query and the images found after query meets the criteria set forth 
in the query metric in which the distance between the queried image and the predetermined 
photography location is within the event boundary. The event boundary for an event occurs 
within a region of interest, for example, a park or a city requires the calculation of the distance 
between the locations for the images being queried and the query location of interest and the 
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distance should be less than a threshold such as the zero threshold for exact query or within a 
predetermined threshold for approximate query in a query language. 

Nevertheless, Wilcock explicitly discloses using threshold for classifying the pictures 
within a location of interest and therefore Wilcok discloses the claim limitation of judging 
whether the distance is over a predetermined threshold value ( See Wilcok column 7, lines 28- 
51). 

Therefore, taking the combined teachings of Loui, Narayanaswami and Wilcock, one of 
the ordinary skill in the art would have incorporated Wilcock' s thresholding into Loui and 
Narayanaswami 's method because these references deal with the image queries based on the 
location of interest using a criteria set forth in the query wherein Wilcock specifically uses a 
threshold number to classify the groups of pictures. One of the ordinary skill in the art would 
have been motivated to use the thresholding to specifically classify the pictures when they fall 
into a number of categories of events ( See Wilcok column 7, lines 28-5 I V 

Re claims 3-5, 8, 10, 13, 15, 18 and 20, Loui teaches in column 8-1 1 adding annotations 
for pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
arranges the picture according to an event or a sub-event wherein the event or sub-event are 
defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some threshold and the event segmentation stage 
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determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value forjudging the distance 
between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries, e.g., the distance is either zero or 
larger than a threshold value so that the photo is judged to be either within or outside the event 
and sub-event boundaries. Loui further teaches in column 7-1 1 the page layout module presents 
to the user an automatically laid out album organized by event with the algorithm choosing the 
albuming parameters by default or specified by the user. Therefore, Loui further discloses the 
claim limitation of "obtaining related data, which is related to the photography location of the at 
least one image data set for which the distance is over the predetermined threshold value, based 
on the location data attached thereto, from a related data storage means which stores a plurality 
of related data sets" wherein the related data set is the metadata including the location of 
photographing stored in the database. Loui further discloses the claim limitation of "generating 
album data, which includes the related data" because Loui discloses displaying album organized 
by event wherein the event are associated with the pictures in the album. 

Re Claims 21-24: Loui, Narayanaswami and Wilcock disclose the query language for the 
images to be classified in a album including the location of interest supplied in a query and 
therefore, the cited reference disclose the predetermined reference position registered as user 
data in a query. 
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Claims 2, 7, 9, 12, 14, 17, and 19 are rejected under 35 U.S.C 103(a) as being 
unpatentable over Loui et al. U.S. Patent No. 6,636,648 (hereinafter Loui), in view of 
Narayanaswami et al. U.S. Patent No. 6,504,571 (hereinafter Narayanaswami) in view of 
Wilcock et al. U.S. Patent No. 6,741,864 (hereinafter Wilcock), in view of Kino et al. U.S. Patent 
No. 6,832,101 (hereinafter Kino). 

Re claims 2, 7, 12, and 17, Loui teaches in column 8-1 1 adding annotations for pictures 
and associating metadata to the pictures. Loui teaches an albuming algorithm that arranges the 
picture according to an event or a sub-event wherein the event or sub-event are defined by the 
metadata including metadata tags and GPS information to determine location and subject depth 
information. The information defines the context surrounding a picture capturing event allowing 
the user to select the set of pictures in the database using an arbitrary query based on the stored 
metadata and the selected set of pictures is filtered by omitting those pictures whose image 
quality metric is below some threshold and the event segmentation stage determines event and 
sub-event boundaries to determine final set of pictures for each event and sub-event. Loui 
classified the pictures using the metadata having the location information and the pictures are 
classified in accordance to the predetermined way of dealing with the actual boundary condition 
of events and sub-events. This classification algorithm according to the event and sub-event 
boundary involves a predetermined threshold value forjudging the distance between the photo 
location contained in the metadata and a reference position associated with an event or sub-event 
such as the event and sub-event boundaries, e.g., the distance is either zero or larger than a 
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non-zero threshold value so that the photo is judged to be either within or outside the event and 
sub-event boundaries. Loui further teaches in column 7-1 1 the page layout module presents to 
the user an automatically laid out album organized by event chronologically or non- 
chronologically with the algorithm choosing the albuming parameters by default or specified by 
the user. The laid out album organized by events arranged in a chronological order requires the 
location information for the events and sub-events along the route. Therefore, Loui further 
discloses the claim limitation of "generating travel route data, which represents a route taken 
during the trip, based on the location data attached to the at least one image data set for which the 
distance is over the predetermined threshold value" because classifying the events and sub- 
events based on the location are travel route data which represents a route taken during a trip for 
locations of the pictures arranged in a chronological order. Loui further discloses generating 
photography data such as the picture data and the metadata which represents that the image data 
set was obtained at the photography locations along the route. 

Loui does not disclose "obtaining a map data set that contains the route from a map 
database which stores a plurality of map data sets, based on the travel route data" and similarly 
"attaching the photography data to the map data set and generating album data including the map 
data set, in which the photography data is correlated with the image data set." 

However, Kino discloses a method of photographing a photo image at a construction side 
to be related with a map and registering it in the database wherein the information about the 
photograph position on the map is stored in the database to be related with attached information 
about the construction site. Kino further discloses associating a symbol to the place where the 
image was photographed on the map allowing for searching the image and the attached 
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information to be related each other by referring to the symbol list using the symbol ID. Kino 
thus discloses the locations of the travel route data chronologically because the locations of the 
pictures taken are ordered chronologically along a travel route. In this way, the locations are 
searched/extracted from the database based on the attached information and the locations thus 
extracted in a chronological order form a travel route (See Kino column 3, lines 50 through 
column 4, lines 16). 

Although Loui did not specifically teach the claim limitation, it is well known to one of 
the ordinary skill in the art to have incorporated Kino's map database to have associated symbols 
or metadata's locations to the photographs and to have determined a travel route based on the 
Loui's location data associated with the pictures. 

One having the ordinary skill in the art would have been motivated to do this because it 
would have provided a travel route on the map database and means for relating the position on 
the map with the symbols attached to or associated with the pictures (See Kino column 3, lines 
50 through column 4, lines 16). 

Re claims 9, 14 and 19, Loui teaches in column 8-1 1 adding annotations for pictures and 
associating metadata to the pictures. Loui teaches an albuming algorithm that arranges the 
picture according to an event or a sub-event wherein the event or sub-event are defined by the 
metadata including metadata tags and GPS information to determine location and subject depth 
information. The information defines the context surrounding a picture capturing event allowing 
the user to select the set of pictures in the database using an arbitrary query based on the stored 
metadata and the selected set of pictures is filtered by omitting those pictures whose image 
quality metric is below some threshold and the event segmentation stage determines event and 
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sub-event boundaries to determine final set of pictures for each event and sub-event. Loui 
classified the pictures using the metadata having the location information and the pictures are 
classified in accordance to the predetermined way of dealing with the actual boundary condition 
of events and sub-events. This classification algorithm according to the event and sub-event 
boundary involves a predetermined threshold value forjudging the distance between the photo 
location contained in the metadata and a reference position associated with an event or sub-event 
such as the event and sub-event boundaries, e.g., the distance is either zero or larger than a 
threshold value so that the photo is judged to be either within or outside the event and sub-event 
boundaries. Loui further teaches in column 7-11 the page layout module presents to the user an 
automatically laid out album organized by event with the algorithm choosing the albuming 
parameters by default or specified by the user. Therefore, Loui further discloses the claim 
limitation of "obtaining related data, which is related to the photography location of the at least 
one image data set for which the distance is over the predetermined threshold value, based on the 
location data attached thereto, from a related data storage means which stores a plurality of 
related data sets" wherein the related data set is the metadata including the location of 
photographing stored in the database. Loui further discloses the claim limitation of "generating 
album data, which includes the related data" because Loui discloses displaying album organized 
by event wherein the event are associated with the pictures in the album. 
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